Operating terminal and operating method

ABSTRACT

An operating terminal operates an external device for projecting and retracting a part of the external device, and includes: a display unit that displays an image; a touch panel on the display unit, and configured to detect a touched area of an object and to receive a position signal in response to the touched area; a determining unit that determines a change of the touched area on the touch panel, based on the position signal; a detecting unit that detects a boundary of areas having different features in the touched area in the image; and a signal generating unit that generates a drive signal indicating a drive amount for causing the part of the external device to be projected and retracted, based on a determination result by the determining unit and a detection result by the detecting unit, and outputs the drive signal to the external device.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is based upon and claims the benefit of priority fromJapanese Patent Application No. 2014-094841, filed on May 1, 2014, theentire contents of which are incorporated herein by reference.

BACKGROUND

1. Technical Field

The disclosure relates to an operating terminal and an operating methodfor operating an external device in which a part of members thereof isadapted to be projected and retracted to be displaced.

2. Related Art

In recent years, many operating terminals with a touch panel provided ona display screen of a display unit that displays images and informationare used. A user performs various operations through the touch panel. Insuch operating terminals, a technology is known, in which the user canperform an operation through the touch panel even he or she is a personwith impaired vision (for example, see Japanese Patent ApplicationLaid-open No. 2012-156607). In this technology, when the display unitdisplays information, and the touch panel is touched, the informationdisplayed at a touched position is converted into Braille and output.

SUMMARY

In some embodiments, an operating terminal operates an external deviceconfigured to project and retract a part of a surface of the externaldevice. The operating terminal includes: a display unit configured todisplay an image; a touch panel provided superimposed on a display areaof the display unit, and configured to detect a touched area of anobject from an outside and to receive a position signal in response tothe touched area; a determining unit configured to determine a change ofthe touched area on the touch panel, based on the position signalreceived by the touch panel; a detecting unit configured to detect aboundary of areas having different features in the touched area in theimage; and a signal generating unit configured to generate a drivesignal that indicates a drive amount for causing the part of the surfaceof the external device to be projected and retracted, based on adetermination result determined by the determining unit and a detectionresult detected by the detecting unit, and to output the drive signal tothe external device.

In some embodiments, an operating method is executed by an operatingterminal for operating an external device configured to project andretract a part of a surface of the external device. The operatingterminal includes: a display unit configured to display an image; and atouch panel provided superimposed on a display area of the display unit,and configured to detect a touched area of an object from an outside andto receive a position signal in response to the touched area. The methodincludes: determining a change of the touched area on the touch panel,based on the position signal received by the touch panel; detecting aboundary of areas having different features in the touched area in theimage; and generating a drive signal that indicates a drive amount forcausing the part of the surface of the external device to be projectedand retracted, based on a determination result in determining the changeof the touched area on the touch panel and a detection result indetecting the boundary of the areas having different features in thetouched area in the image, and outputting the drive signal to theexternal device.

The above and other features, advantages and technical and industrialsignificance of this invention will be better understood by reading thefollowing detailed description of presently preferred embodiments of theinvention, when considered in connection with the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram illustrating a functional configuration of anoperating system according to a first embodiment of the presentinvention;

FIG. 2 is a schematic configuration diagram schematically illustrating across section of an external device according to the first embodiment ofthe present invention;

FIG. 3 is a flowchart illustrating an outline of processing executed byan operating terminal according to the first embodiment of the presentinvention;

FIG. 4 is a diagram schematically illustrating a state in which a usertouches an operating terminal according to the first embodiment of thepresent invention;

FIG. 5A is a diagram schematically illustrating a state in which theuser performs a sliding operation to the operating terminal according tothe first embodiment of the present invention;

FIG. 5B is a diagram schematically illustrating a state in which theuser performs the sliding operation to the operating terminal accordingto the first embodiment of the present invention;

FIG. 6 is a diagram schematically illustrating a state of when a signalgenerating unit of the operating terminal according to the firstembodiment of the present invention generates a drive signal;

FIG. 7A is an operation diagram for schematically describing anoperation of the external device according to the first embodiment ofthe present invention;

FIG. 7B is an operation diagram for schematically describing anoperation of the external device according to the first embodiment ofthe present invention;

FIG. 8 is a schematic configuration diagram schematically illustrating across section of an external device according to a modification of thefirst embodiment of the present invention;

FIG. 9 is a schematic configuration diagram of an operating terminalaccording to a second embodiment of the present invention;

FIG. 10 is a block diagram illustrating a functional configuration ofthe operating terminal according to the second embodiment of the presentinvention;

FIG. 11 is a flowchart illustrating an outline of processing executed bythe operating terminal according to the second embodiment of the presentinvention;

FIG. 12 is a diagram schematically illustrating a state in which a userperforms a sliding operation to the operating terminal according to thesecond embodiment of the present invention;

FIG. 13A is an operation diagram for schematically describing anoperation of a displacement generating unit of the operating terminalaccording to the second embodiment of the present invention;

FIG. 13B is an operation diagram schematically describing an operationof the displacement generating unit of the operating terminal accordingto the second embodiment of the present invention;

FIG. 13C is an operation diagram schematically describing an operationof the displacement generating unit of the operating terminal accordingto the second embodiment of the present invention;

FIG. 14 is a schematic configuration diagram of an imaging systemaccording to a third embodiment of the present invention;

FIG. 15 is a block diagram illustrating a functional configuration ofthe imaging system according to the third embodiment of the presentinvention;

FIG. 16 is a flowchart illustrating an outline of processing executed byan imaging device according to the third embodiment of the presentinvention;

FIG. 17 is a diagram illustrating a state of when an image is capturedusing the imaging device according to the third embodiment of thepresent invention; and

FIG. 18 is a diagram schematically illustrating a state in which a userperforms a sliding operation to the imaging device according to thethird embodiment of the present invention.

DETAILED DESCRIPTION

Modes for carrying out the present invention (hereinafter, referred toas embodiments) will be described below with reference to the drawings.The present invention is not limited by the embodiments below. The samereference signs are used to designate the same elements throughout thedrawings.

First Embodiment

Schematic Configuration of Operating System

FIG. 1 is a block diagram illustrating a functional configuration of anoperating system according to the first embodiment of the presentinvention. An operating system 100 illustrated in FIG. 1 includes anoperating terminal 1 that displays an image and receives an input from auser, and an external device 2 adapted to project and retract a part ofa surface thereof, based on a signal input from the operating terminal1.

Configuration of Operating Terminal

First, a configuration of the operating terminal 1 will be described.

The operating terminal 1 includes an image acquiring unit 10, a displayunit 11, a touch panel 12, a communication unit 13, and a control unit14.

The image acquiring unit 10 images a subject, and acquires (generates)image data of the subject. The image acquiring unit 10 is configured byan optical system that forms a subject image, and an imaging elementsuch as a charge coupled device (CCD) or a complementary metal oxidesemiconductor (CMOS) that receives the subject image formed by theoptical system and performs photoelectric conversion to generate theimage data. Here, the image data includes distance information thatindicates a distance of the subject, luminance information thatindicates brightness, and color information that indicates a color. Notethat the image acquiring unit 10 may acquire the image data from anexternal device or from a recording medium for recording image data.Further, the image acquiring unit 10 may acquire the image data from aserver that records image data, through a network.

The display unit 11 displays an image corresponding to the image dataacquired by the image acquiring unit 10 under control of the controlunit 14. The display unit 11 is configured by a display panel made ofliquid crystal or organic electro luminescence (organic EL), a drivedriver, and the like.

The touch panel 12 is provided superimposed on a display area of thedisplay unit 11. The touch panel 12 detects a touched area by an objectfrom an outside, and receives an input of a position signal according tothe touched area.

The communication unit 13 transmits a drive signal for displacing theexternal device 2 to the external device 2 according to a specifiedcommunication standard, under control of the control unit 14.

The control unit 14 generally controls an operation of the operatingterminal 1 by transferring instructions and data to the respective unitsthat configure the operating terminal 1. The control unit 14 isconfigured by a central processing unit (CPU).

Here, a detailed configuration of the control unit 14 will be described.The control unit 14 includes a display control unit 141, a determiningunit 142, a detecting unit 143, and a signal generating unit 144.

The display control unit 141 controls a display form of the display unit11. To be specific, the display control unit 141 displays, in thedisplay unit 11, the image corresponding to the image data acquired bythe image acquiring unit 10.

The determining unit 142 determines area change of the touched area by auser, on the touch panel 12, based on the position signal, the input ofwhich is received by the touch panel 12. For example, the determiningunit 142 determines a locus of the area change of the touched area onthe touch panel 12. Here, the touched area is an area having a touchedposition as a center.

The detecting unit 143 detects a boundary of areas having differentfeatures in the touched area determined by the determining unit 142, inthe image displayed on the display unit 11. To be specific, thedetecting unit 143 detects respective pieces of distance information ofa plurality of target objects (subjects) included in the imagecorresponding to the image data acquired by the image acquiring unit 10,and detects the distance information and a boundary of areas havingdifferent distances in the touched area determined by the determiningunit 142. In this case, the detecting unit 143 detects the distances ofareas of the target objects, based on image surface phase differencedata (distance information) included in the image data. Note that thedetecting unit 143 may detect not only the distances of the areas of thetarget objects, but also any of the color information, the luminanceinformation, and a contour. In this case, the detecting unit 143 detectslight and darkness of the luminance information, and the contour, byperforming binarization processing for the image data.

The signal generating unit 144 generates the drive signal that indicatesa projection and retraction amount for causing a part of the externaldevice 2 to be projected and retracted, based on a determination resultof the determining unit 142 and a detection result of the detecting unit143, and outputs the drive signal to the external device 2.

Configuration of External Device

Next, a configuration of the external device 2 will be described.

FIG. 2 is a schematic configuration diagram schematically illustrating across section of the external device 2. The external device 2illustrated in FIG. 2 includes a housing 20, a drive unit 21, atransmission unit 22, a pressing member 23, an energizing member 24, asupporting member 25, and a displacement transmitting member 26.

The housing 20 has a cylindrical shape with a bottom, and houses thedrive unit 21, the transmission unit 22, the pressing member 23, theenergizing member 24, the supporting member 25, and the displacementtransmitting member 26 therein.

The drive unit 21 rotates the transmission unit 22 by driving thetransmission unit 22, based on the drive signal input from the operatingterminal 1. The drive unit 21 is configured by a stepping motor, a DCmotor, or the like.

The transmission unit 22 is connected to the drive unit 21, and includesa rotary unit 221 rotated according to driving of the drive unit 21, anda feed screw 222 moved in the right and left direction (see the arrow(a) in FIG. 2) according to the rotation of the rotary unit 221.

The pressing member 23 has an approximately triangular pyramid shape,and is movably provided in a vertical direction along a side wall 201 ofthe housing 20. The pressing member 23 is moved in the verticaldirection by abutting on the feed screw 222.

The energizing member 24 is connected to a bottom part 202 of thehousing 20 and the pressing member 23, and energizes the pressing member23 toward the bottom part 202. The energizing member 24 is configured bya spring or the like.

The supporting member 25 is provided on the pressing member 23, andsupports the displacement transmitting member 26. The supporting member25 has an approximately columnar shape, and is configured by an elasticobject, rubber, or the like.

The displacement transmitting member 26 is movably provided in thevertical direction (see the arrow (b) of FIG. 2) along the side wall 201of the housing 20. The displacement transmitting member 26 protrudesfrom the housing 20 by being pressed by the pressing member 23 throughthe supporting member 25.

The external device 2 configured as described above moves the positionof the feed screw 222 on the rotary unit 221 by driving of the driveunit 21, based on the drive signal input from the operating terminal 1,thereby to move the position of the displacement transmitting member 26so as to be projected and retracted in the vertical direction.

Processing of Operating Terminal

Next, processing executed by the operating terminal 1 will be described.FIG. 3 is a flowchart illustrating an outline of the processing executedby the operating terminal 1.

As illustrated in FIG. 3, the determining unit 142 determines whetherthere has been a touch to the touch panel 12 (step S101). To bespecific, as illustrated in FIG. 4, the determining unit 142 determineswhether the touch panel 12 has been touched. When the determining unit142 determines that there has been a touch to the touch panel 12 (Yes atstep S101), the operating terminal 1 proceeds to step S102. On the otherhand, when the determining unit 142 determines that there has been notouch to the touch panel 12 (No at step S101), the operating terminal 1terminates the present processing.

At step S102, the determining unit 142 determines whether the touchoperation of the user is a sliding operation, and the sliding operationhas passed through a boundary part of the features of the image or thesubject (target object) (step S102). In this case, at the time of thesliding operation by the user, the determining unit 142 may determine atime at which the feature of the image detected by the detecting unit143 is changed. The determining unit 142 may hold a change point of theimage in advance like a database, and determine the time at which afinger is slid from a place not corresponding to the change point to aplace corresponding to the change point. Further, improvement may beadded such that the determination is not performed when the finger isslid along the contour.

For example, as illustrated in FIGS. 5A and 5B, the determining unit 142determines whether the touch operation is the sliding operation in whichthe user moves the finger to the left direction (see the arrow (c))while keeping the touch to the touch panel 12, based on the positionsignal sequentially input through the touch panel 12 (FIG. 5A→FIG. 5B).In this case, the determining unit 142 determines whether the touchoperation of the user is the sliding operation according to the areachange of the touched area (change of a feature of the image of eacharea) sequentially input through the touch panel 12. At this time, thedetecting unit 143 detects a boundary of areas having differentfeatures, for example, one or more of the distance information,luminance components, and color components, in the touched area in theimage displayed on the display unit 11. When the determining unit 142determines that the touch operation of the user is the slidingoperation, and the sliding operation has passed through the boundarypart of the features of the subject (target object) (Yes at step S102),the operating terminal 1 proceeds to step S103 described below. On theother hand, when the determining unit 142 determines that the touchoperation of the user is not the Sliding operation (No at step S102),the operating terminal 1 proceeds to step S105 described below. Notethat the boundary determined by the detecting unit 143 can be detectedonly with the image, and may be determined according to the change ofthe feature of the image. However, other signals such as distribution ofperspective determination using information at the time of auto focusingassociated with each position or area such as coordinates of the imagemay be used together.

At step S103, the determining unit 142 determines whether the areachange of the touched area repeatedly traces the same locus. When thedetermining unit 142 determines that the area change of the touched arearepeatedly traces the same locus (Yes at step S103), the operatingterminal 1 proceeds to step S104 described below. On the other hand,when the determining unit 142 determines that the area change of thetouched area does not repeatedly trace the same locus (No at step S103),the operating terminal 1 proceeds to step S106 described below.

At step S104, the signal generating unit 144 generates the drive signalto increase a projection and retraction amount for causing a part of theexternal device 2 to be projected and retracted, based on thedetermination result determined by the determining unit 142 and thedetection result detected by the detecting unit 143, and outputs thedrive signal to the external device 2. To be specific, as illustrated inFIG. 6, the signal generating unit 144 generates the drive signal thatindicates an increased drive amount, based on the boundary of the areaof a subject K1 detected by the detecting unit 143, for example, theboundary where the distance between a background and the subject K1 ischanged, and the touched area determined by the determining unit 142,and outputs the drive signal to the external device 2. Accordingly, asillustrated in FIGS. 7A and 7B, in the external device 2, the drive unit21 is driven based on the drive signal input from the operating terminal1, so that the displacement transmitting member 26 is raised in a largemanner (FIG. 7A→FIG. 7B). As a result, the user can intuitively graspthe boundary between the subject K1 and the background in the imagedisplayed on the display unit 11.

Following that, when there has been a touch to the touch panel 12 (Yesat step S105), the operating terminal 1 proceeds to step S102. On theother hand, when there has been no touch to the touch panel 12 (No atstep S105), the operating terminal 1 terminates the present processing.

At step S106, the signal generating unit 144 generates the drive signalthat indicates the drive amount for causing a part of the externaldevice 2 to be projected and retracted, based on the determinationresult determined by the determining unit 142 and the detection resultdetected by the detecting unit 143, and outputs the drive signal to theexternal device 2. After step S106, the operating terminal 1 proceeds tostep S105.

According to the first embodiment of the present invention describedabove, the signal generating unit 144 generates the drive signal thatindicates the drive amount for causing a part of the external device 2to be projected and retracted, based on the determination resultdetermined by the determining unit 142 and the detection result detectedby the detecting unit 143 and outputs the drive signal to the externaldevice 2 to causes a part of the external device 2 to be projected andretracted. Therefore, even a user with impaired vision can intuitivelygrasp the image displayed on the display unit 11.

Further, according to the first embodiment of the present invention,when the determining unit 142 determines that the area change of thetouched area repeatedly traces the same locus, the signal generatingunit 144 generates the drive signal that indicates an increased driveamount and outputs the drive signal to the external device 2 to cause apart of the external device 2 to be projected and retracted further.Therefore, the user can further grasp the image displayed on the displayunit 11.

Modification of First Embodiment

FIG. 8 is a schematic configuration diagram schematically illustrating across section of an external device according to a modification of thefirst embodiment. An external device 3 illustrated in FIG. 8 includes ahousing 30, a displacement portion 31, a pressing member 32, asupporting member 33, an energizing member 34, and a displacementtransmitting member 35.

The housing 30 has a cylindrical shape with a bottom, and houses thedisplacement portion 31, the pressing member 32, the supporting member33, the energizing member 34, and the displacement transmitting member35 therein. The housing 30 includes a holding unit 301 therein, whichholds one end of the displacement portion 31.

The displacement portion 31 has a flat plate shape. One end part 311 isheld by the holding unit 301, and the other end part 312 is providedabutting on the pressing member 32. The displacement portion 31 isdisplaced according to the drive signal input from the operatingterminal 1. To be specific, the displacement portion 31 has the otherend part 312 displaced in an up and down direction with the one end part311 as a fulcrum, according to the drive signal input from the operatingterminal 1. The displacement portion 31 is configured by a polymeractuator or bimetal.

The pressing member 32 is provided on a bottom part 30 b of the housing30 through the supporting member 33. The pressing member 32 has one endpart 321 abut on the other end part 312 of the displacement portion 31,and the other end part 322 abut on the displacement transmitting member35.

The energizing member 34 is connected to the pressing member 32 and thebottom part 30 b, and energizes the pressing member 32 toward thedisplacement transmitting member 35. The energizing member 34 isconfigured by a spring or the like.

The displacement transmitting member 35 is movably provided in avertical direction along a side wall 303 of the housing 30. Thedisplacement transmitting member 35 protrudes from the housing 30 bybeing pressed by the pressing member 32.

According to the modification of the first embodiment of the presentinvention as described above, in the external device 3, the displacementportion 31 is displaced based on the drive signal input from theoperating terminal 1, so that the displacement transmitting member 35 ismoved. Therefore, even a user without knowledge about Braille or a userwith impaired vision can intuitively grasp content of the imagedisplayed on the display unit 11 or a composition at the time ofcapturing an image.

Second Embodiment

Next, a second embodiment of the present invention will be described. Inthe second embodiment, an operating terminal has a differentconfiguration from the operating terminal according to the firstembodiment described above. To be specific, the operating terminalaccording to the second embodiment includes a displacement generatingunit including a plurality of external devices according to the firstembodiment. Therefore, hereinafter, a configuration of the operatingterminal according to the second embodiment will be described, and then,processing executed by the operating terminal according to the secondembodiment will be described. The same elements as the operatingterminal 1 and the external device 2 according to the first embodimentare denoted by the same reference signs, and description is omitted.

Configuration of Operating Terminal

FIG. 9 is a schematic configuration diagram of the operating terminalaccording to the second embodiment. FIG. 10 is a block diagramillustrating a functional configuration of the operating terminalaccording to the second embodiment.

An operating terminal 1 a illustrated in FIGS. 9 and 10 includes animage acquiring unit 10, a display unit 11, a touch panel 12, a controlunit 14, and a displacement generating unit 4.

The displacement generating unit 4 includes displacement portions 41 to49. To be specific, the displacement generating unit 4 has thedisplacement portions 41 to 49 arranged in a matrix manner. For example,the displacement generating unit 4 has the displacement portions 41 to49 arranged in 3×3 manner. The displacement portions 41 to 49 have asimilar configuration to the external device 2 or 3 of the firstembodiment, and are projected and retracted according to drive signalsunder control of the control unit 14. Note that the number of thedisplacement portions 41 to 49 is not limited to 9, and can beappropriately changed. For example, 16 (4×4) displacement portions maybe arranged.

Processing of Operating Terminal

Next, processing executed by the operating terminal 1 a will bedescribed. FIG. 11 is a flowchart illustrating an outline of theprocessing executed by the operating terminal 1 a.

As illustrated in FIG. 11, a determining unit 142 determines whetherthere has been a touch to the touch panel 12 (step S201). When thedetermining unit 142 determines that there has been a touch to the touchpanel (Yes at step S201), the operating terminal 1 a proceeds to stepS202 described below. On the other hand, when the determining unit 142determines that there has been no touch to the touch panel 12 (No atstep S201), the operating terminal 1 a terminates the presentprocessing.

At step S202, the determining unit 142 determines whether the touch by auser is a sliding operation. For example, the determining unit 142determines whether the touch is the sliding operation in which the usermoves a finger to a left direction while keeping the touch to the touchpanel 12, as illustrated in FIG. 12. When the determining unit 142determines that the touch by the user is the sliding operation (Yes atstep S202), the operating terminal 1 a proceeds to step S203 describedbelow. On the other hand, when the determining unit 142 determines thatthe touch by the user is not the sliding operation (No at step S202),the operating terminal 1 a proceeds to step S206 described below.

At step S203, the detecting unit 143 determines whether there is aboundary of areas having different features in a slid range where theuser has performed the sliding operation. To be specific, the detectingunit 143 detects a contour in an image by performing specifiedprocessing, for example, binarization processing of a luminance value ofimage data, for the image corresponding to the image data displayed onthe display unit 11, and determines whether the detected contour is inthe slid range. For example, in the case illustrated in FIG. 12, thedetecting unit 143 detects the contour between a background and asubject area of a subject K1, as the boundary. When the detecting unit143 determines that there is a boundary of areas having differentfeatures in the slid range where the user has performed the slidingoperation (Yes at step S203), the operating terminal 1 a proceeds tostep S204 described below. On the other hand, when the detecting unit143 determines that there is no boundary of areas having differentfeatures in the slid range where the user has performed the slidingoperation (No at step S203), the operating terminal 1 a proceeds to stepS206 described below.

At step S204, the detecting unit 143 determines a direction of theboundary of areas having different features of the image, based on adetermination result of a touched area input from the determining unit142. For example, in the case illustrated in FIG. 12, the detecting unit143 determines the direction of the boundary of the subject K1 in thetouched area of an index finger R1 of the user.

Following that, the signal generating unit 144 generates a drive signalof an uneven pattern in the boundary direction, based on thedetermination result determined by the determining unit 142, and theuneven pattern in the boundary direction detected by the detecting unit143, and outputs the drive signal to the displacement generating unit 4(step S205). Accordingly, as illustrated in FIGS. 13A to 13C, the userperforms the sliding operation with the index finger R1, whereby thedisplacement portions 41 to 49 of the displacement generating unit 4respectively are projected and retracted according to the boundary ofareas having different features on the image in the touch (FIG. 13A→FIG.13B→FIG. 13C). Therefore, the user feels the unevenness with an indexfinger L1 of a left hand, thereby to intuitively grasp a composition ofthe image in more detail. Note that, in FIGS. 12, and 13A to 13C, thedisplacement portions 41 to 49 in a protruding state are expressed bybeing filled in black for simplification of the description. With suchimprovement, protruding portions (displacement portions 41 to 49) arearranged corresponding to the direction of change of the boundary ofareas having different features on the image (a diagonal direction fromthe waist line of the person in FIG. 12). Vertical, horizontal, anddiagonal (upward to the right and upward to the left) directions can beexpressed with the nine dots. The determination of the boundary by thedetecting unit 143 may be performed with respect to a color, brightness,a pattern, and the like, for each screen area, and change thereof may bedetermined. While FIG. 12 is illustrated in black and white, in reality,such change of the image feature can be found in a boundary between abackground and a person, or a portion where a hand and a body overlapwith each other in the case of the person, and generally, can berecognized as a contour. Further, feeling of the contour differsdepending on a way of moving a finger. However, if a change ofprojection and retraction is generated based on the same surface as thedisplay screen of the displacement generating unit 4, the sense of touchcorresponding to the change direction can be obtained.

Following that, when there has been a touch to the touch panel 12 (Yesat step S206), the operating terminal 1 a returns the processing to stepS202. On the other hand, when there has been no touch to the touch panel12 (No at step S206), the operating terminal 1 a terminates the presentprocessing.

According to the second embodiment of the present invention describedabove, the signal generating unit 144 generates the drive signal thatindicate a drive amount for causing the displacement generating unit 4to be projected and retracted, based on the determination resultdetermined by the determining unit 142 and the detection result detectedby the detecting unit 143, and outputs the drive signal to thedisplacement generating unit 4 to cause the displacement generating unit4 to be projected and retracted. Therefore, even a user withoutknowledge about Braille or a user with impaired vision can intuitivelygrasp the image displayed on the display unit 11 or the composition atthe time of capturing an image.

Further, according to the second embodiment of the present invention,the signal generating unit 144 generates and outputs the drive signalsof the respective displacement portions 41 to 49. Therefore, thecomposition of the image and the contour of the subject can beintuitively grasped in more detail.

In the second embodiment of the present invention, the displacementgenerating unit 4 and the operating terminal 1 a are integrallyprovided. However, the displacement generating unit 4 and the operatingterminal 1 a may be provided separately. For example, the displacementgenerating unit 4 may be caused to function as an accessory attachableand detachable to and from the operating terminal 1 a.

Third Embodiment

Next, a third embodiment of the present invention will be described. Inthe third embodiment, a case in which the operating terminal accordingto the first embodiment described above is applied to an imaging device.Therefore, hereinafter, a configuration of an imaging device accordingto the third embodiment will be described, and then processing executedby the imaging device according to the third embodiment will bedescribed. The same elements as the operating terminal 1 and theexternal device 2 according to the first embodiment are denoted by thesame reference signs, and description is omitted.

Schematic Configuration of Imaging System

FIG. 14 is a schematic configuration diagram of an imaging systemaccording to the third embodiment. FIG. 15 is a block diagramillustrating a functional configuration of the imaging system accordingto the third embodiment.

An imaging system 200 according to FIGS. 14 and 15 includes an imagingdevice 5 that images a subject to generate image data of the subject,and an external device 2 which is configured to project and retract apart of members thereof so as to be displaced, based on a drive signalinput from the imaging device 5. Note that in the third embodiment, theimaging device 5 functions as an operating terminal.

Configuration of Imaging Device

Next, a configuration of the imaging device 5 will be described.

The imaging device 5 includes a display unit 11, a touch panel 12, acommunication unit 13, an imaging unit 15, a zoom drive unit 16, anoperating unit 17, a recording unit 18, and a control unit 19.

The imaging unit 15 includes an optical system 151 that forms a subjectimage, an imaging element 152 that receives the subject image formed bythe optical system 151 and generates image data by performingphotoelectric conversion, and a lens information recording unit 153 thatrecords lens information related to the optical system 151.

The optical system 151 includes a zoom optical system 151 a and a focusoptical system 151 b. The zoom optical system 151 a changes a focallength of the imaging unit 15 by being moved and extended along anoptical axis O. The focus optical system 151 b adjusts a focus positionof the imaging unit 15 by being moved along the optical axis O.

The zoom drive unit 16 moves the zoom optical system 151 a along theoptical axis O under control of the control unit 19. The zoom drive unit16 is configured by a stepping motor, a DC motor, or the like.

The operating unit 17 receives an input of an instruction signal thatinstructs various operations related to the imaging device 5. Theoperating unit 17 is configured by a power source switch that switches apower source state of the imaging device 5 to an ON state and an OFFstate, a release switch that receives an input of a release signal thatprovides an instruction of capturing an image, a mode switching switchthat switches a mode of the imaging device 5, and the like.

The recording unit 18 records image data input through the control unit19, a program for operating the imaging device 5, and the like. Therecording unit 18 is configured by a synchronous dynamic random accessmemory (SDRAM), a flash memory, a memory card, and the like.

The control unit 19 includes a display control unit 141, a determiningunit 142, a detecting unit 143, a signal generating unit 144, and animage capturing control unit 145.

The image capturing control unit 145 controls capturing of an image bythe imaging unit 15. To be specific, the image capturing control unit145 causes the imaging unit 15 to execute capturing of an image when therelease signal is input from the operating unit 17.

Processing of Imaging Device

Next, processing executed by the imaging device 5 will be described.FIG. 16 is a flowchart illustrating an outline of processing executed bythe imaging device 5.

First, as illustrated in FIG. 16, reference will be made to a case inwhich the imaging device 5 is set to a shooting mode (Yes at step S301).In this case, the display control unit 141 displays a live view imagecorresponding to image data generated by the imaging unit 15, in thedisplay unit 11 (step S302). To be specific, as illustrated in FIG. 17,the user captures an image while placing a thumb on the external device2 and holding the imaging system 200 with the left hand.

Following that, the detecting unit 143 determines a feature of the liveview image (step S303). To be specific, the detecting unit 143 detects aboundary of areas having different features, for example, brightness, inthe live view image, based on live view image data. For example, thedetecting unit 143 detects the boundary between the sky and the groundin the live view image, based on the live view image data.

Following that, when the external device 2 as an accessory is connectedto the communication unit 13 of the imaging device 5 (Yes at step S304),the control unit 19 acquires accessory information from the externaldevice 2 (step S305). After step S305, the imaging device 5 proceeds tostep S306 described below. On the other hand, when the external device 2as an accessory is not connected to the communication unit 13 of theimaging device 5 (No at step S304), the imaging device 5 proceeds tostep S309 described below.

At step S306, when there has been a sliding operation to the touch panel12 (Yes at step S306), and when the detecting unit 143 has detected theboundary of areas having different features on a locus of the slidingoperation (Yes at step S307), the signal generating unit 144 generates adrive signal that indicates a drive amount for causing a part of theexternal device 2 to be projected and retracted, based on adetermination result determined by the determining unit 142 and adetection result detected by the detecting unit 143, and outputs thedrive signal to the external device 2 (step S308). Accordingly, asillustrated in FIG. 18, when the user performs the sliding operation(see the arrow (f) in FIG. 18) to the image displayed on the displayunit 11, a part of the external device 2 is projected and retracted,whereby the user can intuitively grasp the boundary between an area of asubject K1 and a background area on the image. Further, the user cangrasp an area or a contour of the subject K1 while holding a finger onthe operating unit 17. Therefore, the user can be prevented from missinga photo opportunity.

Following that, when there has been a release signal from the operatingunit 17 (Yes at step S309), the image capturing control unit 145 causesthe imaging unit 15 to execute capturing of an image (step S310), andrecords the image data generated by the imaging unit 15, in therecording unit 18 (step S311).

Following that, when the power source is turned OFF by an operation ofthe operating unit 17 (Yes at step S312), the imaging device 5terminates the present processing. On the other hand, when the powersource is not turned OFF by the operation of the operating unit 17 (Noat step S312), the imaging device 5 returns the processing to step S301.

At step S306, when there has been no sliding operation to the touchpanel 12 (No at step S306), the imaging device 5 proceeds to step S309.

At step S307, when the boundary of areas having different features isnot detected on the locus of the sliding operation by the detecting unit143 (No at step S307), the imaging device 5 proceeds to step S309.

At step S309, when there has been no release signal from the operatingunit 17 (No at step S309), the imaging device 5 proceeds to step S312.

Next, reference will be made to a case where the imaging device 5 is setto a playback mode (Yes at step S313) when the imaging device 5 is notset to the shooting mode (No at step S301). At this time, the displaycontrol unit 141 displays, in the display unit 11, a list of imagesrespectively corresponding to a plurality of image data recorded in therecording unit 18 (step S314).

Following that, when an image has been selected through the touch panel12 or the operating unit 17 from among the plurality of images displayedon the display unit 11 (Yes at step S315), the display control unit 141enlarges and displays the selected image in the display unit 11 (stepS316).

Following that, the detecting unit 143 detects a feature of the imageenlarged and displayed on the display unit 11 (step S317).

Following that, when there has been a sliding operation to the touchpanel 12 (Yes at step S318), the imaging device 5 proceeds to the stepS319 described below. On the other hand, when there has been no slidingoperation to the touch panel 12 (No at step S318), the imaging device 5proceeds to step S322 described below.

At step S319, the control unit 19 acquires the lens information from thelens information recording unit 153 of the imaging unit 15.

Following that, when the detecting unit 143 has detected that there is aboundary of areas having different features on the locus of the slidingoperation (Yes at step S320), the image capturing control unit 145drives the zoom drive unit 16 according to the drive signal generated bythe signal generating unit 144, based on the determination resultdetermined by the determining unit 142 and the detection result detectedby the detecting unit 143, to execute zoom processing including zoom upthat extends the zoom optical system 151 a or zoom down that retractsthe zoom optical system 151 a (step S321). Accordingly, the user canintuitively grasp the unevenness of the image by the extension orretraction of the zoom optical system 151 a from or into the housing ofthe imaging device 5.

Following that, when there has been change of the image displayed on thedisplay unit 11 through the operating unit 17 (Yes at step S322), theimaging device 5 returns the processing to step S316. On the other hand,when there has been no change of the image displayed on the display unit11 through the operating unit 17 (No at step S322), the imaging device 5proceeds to step S312.

At step S320, when the detecting unit 143 has determined that there hasbeen no change in the feature of the image on the locus of the slidingoperation (No at step S320), the imaging device 5 proceeds to step S322.

At step S313, when the imaging device 5 is not set to the playback mode(No at step S313), the imaging device 5 proceeds to step S312.

At step S315, when no image has been selected through the touch panel 12or the operating unit 17 from among the plurality of images displayed onthe display unit 11 (No at step S315), the imaging device 5 proceeds tostep S312.

According to the third embodiment of the present invention describedabove, the signal generating unit 144 generates the drive signal thatindicates the drive amount for displacing the external device 2, basedon the determination result determined by the determining unit 142 andthe detection result detected by the detecting unit 143, and outputs thedrive signal to the external device 2 to cause a part of the externaldevice 2 to be projected and retracted. Therefore, even a user withoutknowledge about Braille or a user with impaired vision can intuitivelygrasp the boundary of areas having different features in the imagedisplayed on the display unit 11, or a schematic composition at the timeof capturing an image.

Other Embodiments

An imaging device according to the present invention can be applied toelectronic devices such as a digital camera or a digital video camera towhich an accessory or the like can be mounted, and a mobile phone or atablet-type mobile device having an imaging function, other than adigital single-lens reflex camera. If such user assistance that appealsto a user's sense (here, a sense of touch) is performed, auxiliaryinformation other than visual confirmation can be obtained in industrialdevices or medical devices, and more reliable operations than a simpletouch or sliding, and its determination can be possible. It is notnecessary to record an imaging result like a camera, and this technologymay be used at the time of observation. Therefore, this terminal may beused as an observation device that effectively uses the user's sense,and such an idea is effective for user assistance such as a blindoperation.

Programs executed by the operating terminal and the imaging deviceaccording to the present invention are recorded in a computer-readablerecording medium such as a CD-ROM, a flexible disk (FD), a CD-R, adigital versatile disk (DVD), a USB medium, or flash memory, in filedata in an installable format or executable format.

Further, the programs executed by the operating terminal and the imagingdevice according to some embodiments may be provided by being stored ona computer connected to a network such as the Internet, and downloadedthrough the network. The programs executed by the operating terminal andthe imaging device according to some embodiments may be provided ordistributed through the network such as the Internet.

Note that, in the description of the flowcharts in the presentspecification, the context of the processing among the steps has beenclearly indicated using the expressions such as “first”, “then”, and“following that”. However, the order of the processing necessary forimplementing the present invention is not uniquely determined by suchexpressions. That is, the order of the processing in the flowchartsdescribed in the present specification can be changed within aconsistent range.

The present invention may include various embodiments that are notdescribed here, and various design changes and the like can be performedwithin the scope of the technical idea specified by claims.

Additional advantages and modifications will readily occur to thoseskilled in the art. Therefore, the invention in its broader aspects isnot limited to the specific details and representative embodiments shownand described herein. Accordingly, various modifications may be madewithout departing from the spirit or scope of the general inventiveconcept as defined by the appended claims and their equivalents.

What is claimed is:
 1. An operating terminal for operating an externaldevice configured to project and retract a part of a surface of theexternal device, the operating terminal comprising: a display configuredto display an image; a touch panel provided superimposed on a displayarea of the display, and configured to detect a touched area of anobject from an outside and to receive a position signal in response tothe touched area; and a processor executing instructions to: determine achange of the touched area on the touch panel by determining a path ofthe touched area, based on the position signal received by the touchpanel; detect a boundary of areas having different features in thetouched area in the image; and generate a drive signal that indicates adrive amount for causing the part of the surface of the external deviceto be projected and retracted, based on a determination result and adetection result, and to output the drive signal to the external device;wherein a relative intensity of displacement on the surface of theexternal device caused by the projection and retraction is modulated bythe drive signal in response to the path of the touched area.
 2. Theoperating terminal according to claim 1, wherein when it is determinedthat the touched area repeatedly traces a same path, the processorgenerates the drive signal for increasing the drive amount, and outputsthe drive signal to the external device.
 3. The operating terminalaccording to claim 1, wherein the external device includes a pluralityof members, and is configured to project and retract the plurality ofmembers, respectively, and the processor is configured to generate drivesignals for respectively causing the plurality of members to beprojected and retracted, according to a direction of a change of theboundary of the areas having different features, and to output the drivesignals to the external device.
 4. The operating terminal according toclaim 1, further comprising a communication unit which is configured tocommunicate with the external device, and to which the external deviceis detachably connected.
 5. The operating terminal according to claim 1,further comprising: an image acquisition device configured to image asubject to generate image data of the subject; and the processor isconfigured to display, on the display, an image corresponding to theimage data generated by the image acquisition device.
 6. The operatingterminal according to claim 1, wherein the features are one or more ofdistance information, luminance information, and color information of asubject included in the image.
 7. The operating terminal according toclaim 1, wherein the part of the surface of the external device isprojectable and retractable in response to the generated drive signal.8. The operating terminal according to claim 1, wherein the intensity ofdisplacement on the surface of the external device caused by theprojection and retraction is increased by the drive signal when the pathof the touched area is repeatedly traced.
 9. An operating methodexecuted by an operating terminal for operating an external deviceconfigured to project and retract a part of a surface of the externaldevice, the operating terminal comprising: a display configured todisplay an image; and a touch panel provided superimposed on a displayarea of the display, and configured to detect a touched area of anobject from an outside and to receive a position signal in response tothe touched area, the method comprising: determining a change of thetouched area on the touch panel by determining a path of the touchedarea, based on the position signal received by the touch panel;detecting a boundary of areas having different features in the touchedarea in the image; and generating a drive signal that indicates a driveamount for causing the part of the surface of the external device to beprojected and retracted, based on a determination result in determiningthe change of the touched area on the touch panel and a detection resultin detecting the boundary of the areas having different features in thetouched area in the image, and outputting the drive signal to theexternal device; wherein a relative intensity of displacement on thesurface of the external device caused by the projection and retractionis modulated by the drive signal in response to the path of the touchedarea.
 10. The operating method according to claim 9, wherein the part ofthe surface of the external device is projected and retracted inresponse to the generated drive signal.
 11. The operating methodaccording to claim 9, wherein the intensity of displacement on thesurface of the external device caused by the projection and retractionis increased by the drive signal when the path of the touched area isrepeatedly traced.